双活数据中心

场景说明

随着技术的发展，灾备要求的提高，如何建设RPO=0，RTO≈0的双活解决方案成为数据中心建设选型的第一选择。双活解决方案从其实现双活的层次上区分主要有三种：

  • 存储级双活：落盘存储的数据双向同步，典型的解决方案有延展集群和SAN存储双活。
  • 平台级双活：控制面和数据面双冗余，单个数据中心故障，不影响平台管理，云主机异地恢复。
  • 业务应用级双活：业务应用双向同步，有状态业务：如Oracle RAC、MySQL主从复制；无状态的业务如Web等。

ZStack Cloud云平台结合多种双活的特点提供两种双活解决方案：存储+平台+应用双活和平台+应用双活。两种解决方案对比如下：

image-20220805142601424

本章节以存储+平台+应用双活解决方案为例，提供云平台、分布式存储以及整体网络的基本规划方法和配置举例（应用双活、全局负载均衡和全局路由高可用设计不在此处讨论）。

  • 平台规划：
    • 底层使用延展存储池对接作为ZStack Cloud云平台的主存储、镜像存储，加载到两个集群
    • 镜像服务器建议采用Ceph类型
    • 业务网络需打通
    • 关键业务的云主机，设置NeverStop标签
    • 多使用VPC高可用、LB、弹性伸缩等技术，提高系统冗余
    • 站点间使用裸纤或专线连接，站点内二层互通，站点与仲裁节点间可以三层互通
  • 高级设置：
    • 云主机跨集群高可用开关开启（默认关闭）
  • 云平台部署要求：
    • 管理网络：延迟5ms以内（单向2.5ms），带宽1Gbps及以上
    • 业务网络：二层网络（业务网络），业务网络需打通
    • 主存储：存储池添加延展集群存储池，同时加载到两个集群
    • 服务器配比：站点内计算节点、存储节点建议1:1配置，避免木桶效应
    • 镜像存储池：高冗余&高性能：延展池；中冗余&高性能：普通池；低冗余&低成本：镜像仓库
  • 分布式存储部署要求：
    • 距离限制
      • 数据中心间建议小于100公里
    • 延迟要求
      • 站点间RTT≤5ms
      • 站点与仲裁RTT≤100ms
    • 带宽要求
      • 站点间带宽≥10Gbps，建议20Gbps及以上；
      • 站点与仲裁间带宽≥2Gbps
    • MTU设置
      • 站点间MTU设置要保持一致，可设置为MTU=9000

场景拓扑

image-20220812172856055

连接示例

  • 核心交换机端口互连表

image-20220812145431108

image-20220812145455447

  • 管理交换机端口互连表

image-20220812144634786

image-20220812144714675

  • 业务交换机端口互连表

image-20220812144759306

image-20220812144821748

  • 存储交换机端口互连表

image-20220812144847018

image-20220812144913990

  • BMC交换机端口互连表

image-20220812145542123

image-20220812145610383

  • 仲裁节点及站点互连端口互连表

image-20220812145651153

网络规划

请根据服务器数量、云主机数量规划IP&VLAN&VXLAN，建议预留冗余量便于扩容。

  • VLAN规划

    • 管理：2-89
      • 物理机管理：10
      • VPC路由器管理：20
      • 物理机BMC：30
      • 交换机带外管理：40
    • 存储：90-99
    • 私有：100-1999（扁平网络+VPC网络）
    • 公有：2000-2999
    • DC互连：4000-4010

  • VXLAN规划

    • VXLAN Pool：5000-59999

  • IP规划

    • 管理网络IP：192.168.224.0/20（16个C）
      • 物理机管理：192.168.224.0/24
      • VPC路由器管理：192.168.228.0/24
      • 物理机BMC：192.168.232.0/24
      • 交换机带外管理：192.168.236.0/24
    • 存储网络IP：192.168.252.0/22（4个C）
    • 公有网络IP：172.31.0.0/20（16个C）
    • 私有网络IP：10.255.0.0/16（256个C）
    • 仲裁站点IP：192.168.220.0/22（4个C）
    • DC互连IP：192.168.200.0/24（1个C）

  • 生成树规划

    • 此网络作为独立二层网络时建议把Spine配置为根桥；如果此网络二层接入现网，则需与网络运维工程师共同规划根桥，不可私自指定根桥，避免发生生成树抢根导致网络中断。
    • 一般情况下建议所有与服务器互连的交换机端口配置成边缘端口，可以加速生成树收敛、提高网络稳定性。

配置举例

下面以服务器服务器bond-lacp、交换机堆叠为例提供服务器网络配置和交换机配置参考。

SiteA 配置参考：

############## 核心交换机A-Spine1（华为CE12804） ################
#堆叠配置请参考厂商产品文档
#配置交换机名称
sysname A-Spine1
#配交换机带外管理
ip vpn-instance mgmt
 ipv4-family
#
interface MEth0/0/0
 ip binding vpn-instance mgmt
 ip address 192.168.236.1 255.255.255.0
#
ip route-static vpn-instance mgmt 0.0.0.0 0.0.0.0 192.168.236.254    
#配置生成树
stp root primary
stp tc-protection
stp bpdu-protection
#
#配置管理VLAN
vlan batch 10 20 30 40
#
vlan 10
 description == Mgmt-Server ==
#
vlan 20
 description == Mgmt-VPC-Router ==
#
vlan 30
 description == Mgmt-Server-BMC ==
#
vlan 40
 description == Mgmt-Switch-BMC ==
#
interface Vlanif10
 ip address 192.168.224.251 24
 vrrp vrid 1 virtual-ip 192.168.224.254
 vrrp vrid 1 priority 120
 vrrp vrid 1 preempt timer delay 20
#
interface Vlanif20
 ip address 192.168.228.251 24
 vrrp vrid 1 virtual-ip 192.168.228.254
 vrrp vrid 1 priority 120
 vrrp vrid 1 preempt timer delay 20
#
interface Vlanif30
 ip address 192.168.232.251 24
 vrrp vrid 1 virtual-ip 192.168.232.254
 vrrp vrid 1 priority 120
 vrrp vrid 1 preempt timer delay 20
#
interface Vlanif40
 ip address 192.168.236.251 24
 vrrp vrid 1 virtual-ip 192.168.236.254
 vrrp vrid 1 priority 120
 vrrp vrid 1 preempt timer delay 20
#配置业务VLAN
vlan batch 100 to 2999
#
vlan 100
 description == Private-1 ==
#
vlan 101
 description == Private-2 ==
#
vlan 2000
 description == Public-1 ==
#
vlan 2001
 description == Public-2 ==
#
interface Vlanif2000
 ip address 172.31.0.251 24
 vrrp vrid 1 virtual-ip 172.31.0.254
 vrrp vrid 1 priority 120
 vrrp vrid 1 preempt timer delay 20
#
interface Vlanif2001
 ip address 172.31.1.251 24
 vrrp vrid 1 virtual-ip 172.31.1.254
 vrrp vrid 1 priority 120
 vrrp vrid 1 preempt timer delay 20
#配置存储VLAN
vlan batch 90
#
vlan 90
 description == Storage ==
#
interface Vlanif90
 ip address 192.168.252.251 24
 vrrp vrid 1 virtual-ip 192.168.252.254
 vrrp vrid 1 priority 120
 vrrp vrid 1 preempt timer delay 20
#配置下联业务接入交换机A-Mgmt-Leaf1端口
interface Eth-Trunk1
 description == To_A-Mgmt-Leaf1 ==
 mode lacp-static
 trunkport 100GE 1/1/0/1
 trunkport 100GE 2/1/0/1
 trunkport 100GE 1/2/0/1
 trunkport 100GE 2/2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 10 20
#
#配置下联业务接入交换机A-Biz-Leaf1端口
interface Eth-Trunk1
 description == To_A-Biz-Leaf1 ==
 mode lacp-static
 trunkport 100GE 1/1/0/2
 trunkport 100GE 2/1/0/2
 trunkport 100GE 1/2/0/2
 trunkport 100GE 2/2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
#
#配置下联业务接入交换机A-Stor-Leaf1端口
interface Eth-Trunk1
 description == To_A-Stor-Leaf1 ==
 mode lacp-static
 trunkport 100GE 1/1/0/3
 trunkport 100GE 2/1/0/3
 trunkport 100GE 1/2/0/3
 trunkport 100GE 2/2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90 to 99
#
#配置下联业务接入交换机A-BMC-Leaf1端口
interface Eth-Trunk1
 description == To_A-BMC-Leaf1 ==
 mode lacp-static
 trunkport 100GE 1/1/0/4
 trunkport 100GE 2/1/0/4
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 30 40
#
############## 管理接入交换机A-Mgmt-Leaf1（华为CE6881-48S6CQ） ################
#堆叠配置请参考官方产品文档
#配置交换机名称
sysname A-Mgmt-Leaf1
#配交换机带外管理
ip vpn-instance mgmt
 ipv4-family
#
interface MEth0/0/0
 ip binding vpn-instance mgmt
 ip address 192.168.236.2 255.255.255.0
#
ip route-static vpn-instance mgmt 0.0.0.0 0.0.0.0 192.168.236.254
#配置生成树
stp tc-protection
stp bpdu-protection
#配置管理VLAN
vlan batch 10 20 30 40
#
vlan 10
 description == Mgmt-Server ==
#
vlan 20
 description == Mgmt-VPC-Router ==
#
#配置上联核心A-Spine1端口
interface Eth-Trunk100
 description == To_A-Spine1 ==
 mode lacp-static
 trunkport 100GE 1/0/1
 trunkport 100GE 1/0/3
 trunkport 100GE 2/0/1
 trunkport 100GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 10 20
#配置下联服务器a-zstack-1管理端口
interface Eth-Trunk1
 description == To_a-zstack-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#配置下联服务器a-zstack-2管理端口
interface Eth-Trunk2
 description == To_a-zstack-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2 
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#配置下联服务器a-zstack-3管理端口
interface Eth-Trunk3
 description == To_a-zstack-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#
#配置下联服务器a-ceph-1管理端口
interface Eth-Trunk1
 description == To_a-ceph-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#配置下联服务器a-ceph-2管理端口
interface Eth-Trunk2
 description == To_a-ceph-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2 
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#配置下联服务器a-ceph-3管理端口
interface Eth-Trunk3
 description == To_a-ceph-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#

############## 业务接入交换机A-Biz-Leaf1（华为CE6881-48S6CQ） ################
#堆叠配置请参考官方产品文档
#配置交换机名称
sysname A-Biz-Leaf1
#配交换机带外管理
ip vpn-instance mgmt
 ipv4-family
#
interface MEth0/0/0
 ip binding vpn-instance mgmt
 ip address 192.168.236.3 255.255.255.0
#
ip route-static vpn-instance mgmt 0.0.0.0 0.0.0.0 192.168.236.254
#配置生成树
stp tc-protection
stp bpdu-protection
#配置业务VLAN
vlan batch 100 101 2000 2001
#
vlan 100
 description == Private-1 ==
#
vlan 10
 description == Private-2 ==
#
vlan 2000
 description == Public-1 ==
#
vlan 2001
 description == Public-2 ==
#配置上联核心A-Spine1端口
interface Eth-Trunk100
 description == To_A-Spine1 ==
 mode lacp-static
 trunkport 100GE 1/0/1
 trunkport 100GE 1/0/3
 trunkport 100GE 2/0/1
 trunkport 100GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
#配置下联服务器a-zstack-1业务端口
interface Eth-Trunk1
 description == To_a-zstack-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
 stp edged-port enable
#配置下联服务器a-zstack-2业务端口
interface Eth-Trunk2
 description == To_a-zstack-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
 stp edged-port enable
#配置下联服务器a-zstack-3业务端口
interface Eth-Trunk3
 description == To_a-zstack-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
 stp edged-port enable
#

############## 存储接入交换机A-Stor-Leaf1（华为CE6881-48S6CQ） ################
#堆叠配置请参考官方产品文档
#配置交换机名称
sysname A-Stor-Leaf1
#配交换机带外管理
ip vpn-instance mgmt
 ipv4-family
#
interface MEth0/0/0
 ip binding vpn-instance mgmt
 ip address 192.168.236.4 255.255.255.0
#
ip route-static vpn-instance mgmt 0.0.0.0 0.0.0.0 192.168.236.254
#配置生成树
stp tc-protection
stp bpdu-protection
#配置业务VLAN
vlan 90
 description == Storage ==
#配置下联服务器a-zstack-1存储端口
interface Eth-Trunk1
 description == To_a-zstack-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#配置下联服务器a-zstack-2存储端口
interface Eth-Trunk2
 description == To_a-zstack-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#配置下联服务器a-zstack-3存储端口
interface Eth-Trunk3
 description == To_a-zstack-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#
#配置下联服务器a-ceph-1存储端口
interface Eth-Trunk1
 description == To_a-ceph-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#配置下联服务器a-ceph-2存储端口
interface Eth-Trunk2
 description == To_a-ceph-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#配置下联服务器a-ceph-3存储端口
interface Eth-Trunk3
 description == To_a-ceph-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#

############## BMC接入交换机A-BMC-Leaf1（华为CE6881-48S6CQ） ################
#配置交换机名称
sysname A-BMC-Leaf1
#配置BMC管理VLAN
vlan batch 30 40
#
vlan 30
 description == Mgmt-Server-BMC ==
#
vlan 40
 description == Mgmt-Switch-BMC ==
#配BMC交换机带内管理
interface Vlanif40
 ip address 192.168.236.201 24
#
ip route-static 0.0.0.0 0.0.0.0 192.168.236.254
#
#配置上联核心A-Mgmt-Spine1端口
interface Eth-Trunk100
 description == To_A-Mgmt-Spine1 ==
 mode lacp-static
 trunkport 100GE 1/0/1
 trunkport 100GE 1/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 30 40
#
interface GE1/0/1
 description == To_A-Spine1A == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/2
 description == To_A-Spine1B == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/3
 description == To_A-Mgmt-Leaf1A == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/4
 description == To_A-Mgmt-Leaf1B == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/5
 description == To_A-Biz-Leaf1A == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/6
 description == To_A-Biz-Leaf1B == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/7
 description == To_A-Stor-Leaf1A == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/8
 description == To_A-Stor-Leaf1B == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/9
 description == To_a-zstack-1 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/10
 description == To_a-zstack-2 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/11
 description == To_a-zstack-3 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/12
 description == To_a-ceph-1 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/13
 description == To_a-ceph-2 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/14
 description == To_a-ceph-3 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
#

SiteB配置参考：

############## 核心交换机B-Spine1（华为CE12804） ################
#堆叠配置请参考厂商产品文档
#配置交换机名称
sysname B-Spine1
#配交换机带外管理
ip vpn-instance mgmt
 ipv4-family
#
interface MEth0/0/0
 ip binding vpn-instance mgmt
 ip address 192.168.236.101 255.255.255.0
#
ip route-static vpn-instance mgmt 0.0.0.0 0.0.0.0 192.168.236.254    
#配置生成树
stp root secondary
stp tc-protection
stp bpdu-protection
#
#配置管理VLAN
vlan batch 10 20 30 40
#
vlan 10
 description == Mgmt-Server ==
#
vlan 20
 description == Mgmt-VPC-Router ==
#
vlan 30
 description == Mgmt-Server-BMC ==
#
vlan 40
 description == Mgmt-Switch-BMC ==
#
interface Vlanif10
 ip address 192.168.224.252 24
 vrrp vrid 1 virtual-ip 192.168.224.254
 vrrp vrid 1 priority 100
#
interface Vlanif20
 ip address 192.168.228.252 24
 vrrp vrid 1 virtual-ip 192.168.228.254
 vrrp vrid 1 priority 100
#
interface Vlanif30
 ip address 192.168.232.252 24
 vrrp vrid 1 virtual-ip 192.168.232.254
 vrrp vrid 1 priority 100
#
interface Vlanif40
 ip address 192.168.236.252 24
 vrrp vrid 1 virtual-ip 192.168.236.254
 vrrp vrid 1 priority 100
#配置业务VLAN
vlan batch 100 to 2999
#
vlan 100
 description == Private-1 ==
#
vlan 101
 description == Private-2 ==
#
vlan 2000
 description == Public-1 ==
#
vlan 2001
 description == Public-2 ==
#
interface Vlanif2000
 ip address 172.31.0.252 24
 vrrp vrid 1 virtual-ip 172.31.0.254
 vrrp vrid 1 priority 100
#
interface Vlanif2001
 ip address 172.31.1.252 24
 vrrp vrid 1 virtual-ip 172.31.1.254
 vrrp vrid 1 priority 100
#配置存储VLAN
vlan batch 90
#
vlan 90
 description == Storage ==
#
interface Vlanif90
 ip address 192.168.252.252 24
 vrrp vrid 1 virtual-ip 192.168.252.254
 vrrp vrid 1 priority 100
#配置下联业务接入交换机B-Mgmt-Leaf1端口
interface Eth-Trunk1
 description == To_B-Mgmt-Leaf1 ==
 mode lacp-static
 trunkport 100GE 1/1/0/1
 trunkport 100GE 2/1/0/1
 trunkport 100GE 1/2/0/1
 trunkport 100GE 2/2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 10 20
#
#配置下联业务接入交换机B-Biz-Leaf1端口
interface Eth-Trunk1
 description == To_B-Biz-Leaf1 ==
 mode lacp-static
 trunkport 100GE 1/1/0/2
 trunkport 100GE 2/1/0/2
 trunkport 100GE 1/2/0/2
 trunkport 100GE 2/2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
#
#配置下联业务接入交换机B-Stor-Leaf1端口
interface Eth-Trunk1
 description == To_B-Stor-Leaf1 ==
 mode lacp-static
 trunkport 100GE 1/1/0/3
 trunkport 100GE 2/1/0/3
 trunkport 100GE 1/2/0/3
 trunkport 100GE 2/2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90 to 99
#
#配置下联业务接入交换机B-BMC-Leaf1端口
interface Eth-Trunk1
 description == To_B-BMC-Leaf1 ==
 mode lacp-static
 trunkport 100GE 1/1/0/4
 trunkport 100GE 2/1/0/4
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 30 40
#
############## 管理接入交换机B-Mgmt-Leaf1（华为CE6881-48S6CQ） ################
#堆叠配置请参考官方产品文档
#配置交换机名称
sysname B-Mgmt-Leaf1
#配交换机带外管理
ip vpn-instance mgmt
 ipv4-family
#
interface MEth0/0/0
 ip binding vpn-instance mgmt
 ip address 192.168.236.102 255.255.255.0
#
ip route-static vpn-instance mgmt 0.0.0.0 0.0.0.0 192.168.236.254
#配置生成树
stp tc-protection
stp bpdu-protection
#配置管理VLAN
vlan batch 10 20 30 40
#
vlan 10
 description == Mgmt-Server ==
#
vlan 20
 description == Mgmt-VPC-Router ==
#
#配置上联核心B-Spine1端口
interface Eth-Trunk100
 description == To_B-Spine1 ==
 mode lacp-static
 trunkport 100GE 1/0/1
 trunkport 100GE 1/0/3
 trunkport 100GE 2/0/1
 trunkport 100GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 10 20
#配置下联服务器b-zstack-1管理端口
interface Eth-Trunk1
 description == To_b-zstack-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#配置下联服务器b-zstack-2管理端口
interface Eth-Trunk2
 description == To_b-zstack-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2 
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#配置下联服务器b-zstack-3管理端口
interface Eth-Trunk3
 description == To_b-zstack-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#
#配置下联服务器b-ceph-1管理端口
interface Eth-Trunk1
 description == To_b-ceph-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#配置下联服务器b-ceph-2管理端口
interface Eth-Trunk2
 description == To_b-ceph-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2 
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#配置下联服务器b-ceph-3管理端口
interface Eth-Trunk3
 description == To_b-ceph-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 2
 port trunk pvid vlan 2
 stp edged-port enable
#
############## 业务接入交换机B-Biz-Leaf1（华为CE6881-48S6CQ） ################
#堆叠配置请参考官方产品文档
#配置交换机名称
sysname B-Biz-Leaf1
#配交换机带外管理
ip vpn-instance mgmt
 ipv4-family
#
interface MEth0/0/0
 ip binding vpn-instance mgmt
 ip address 192.168.236.103 255.255.255.0
#
ip route-static vpn-instance mgmt 0.0.0.0 0.0.0.0 192.168.236.254
#配置生成树
stp tc-protection
stp bpdu-protection
#配置业务VLAN
vlan batch 100 101 2000 2001
#
vlan 100
 description == Private-1 ==
#
vlan 10
 description == Private-2 ==
#
vlan 2000
 description == Public-1 ==
#
vlan 2001
 description == Public-2 ==
#配置上联核心B-Spine1端口
interface Eth-Trunk100
 description == To_B-Spine1 ==
 mode lacp-static
 trunkport 100GE 1/0/1
 trunkport 100GE 1/0/3
 trunkport 100GE 2/0/1
 trunkport 100GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
#配置下联服务器b-zstack-1业务端口
interface Eth-Trunk1
 description == To_b-zstack-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
 stp edged-port enable
#配置下联服务器b-zstack-2业务端口
interface Eth-Trunk2
 description == To_b-zstack-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
 stp edged-port enable
#配置下联服务器b-zstack-3业务端口
interface Eth-Trunk3
 description == To_b-zstack-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 100 to 2999
 stp edged-port enable
#
############## 存储接入交换机B-Stor-Leaf1（华为CE6881-48S6CQ） ################
#堆叠配置请参考官方产品文档
#配置交换机名称
sysname B-Stor-Leaf1
#配交换机带外管理
ip vpn-instance mgmt
 ipv4-family
#
interface MEth0/0/0
 ip binding vpn-instance mgmt
 ip address 192.168.236.104 255.255.255.0
#
ip route-static vpn-instance mgmt 0.0.0.0 0.0.0.0 192.168.236.254
#配置生成树
stp tc-protection
stp bpdu-protection
#配置业务VLAN
vlan 90
 description == Storage ==
#配置下联服务器b-zstack-1存储端口
interface Eth-Trunk1
 description == To_b-zstack-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#配置下联服务器b-zstack-2存储端口
interface Eth-Trunk2
 description == To_b-zstack-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#配置下联服务器b-zstack-3存储端口
interface Eth-Trunk3
 description == To_b-zstack-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#
#配置下联服务器b-ceph-1存储端口
interface Eth-Trunk1
 description == To_b-ceph-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#配置下联服务器b-ceph-2存储端口
interface Eth-Trunk2
 description == To_b-ceph-2 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#配置下联服务器b-ceph-3存储端口
interface Eth-Trunk3
 description == To_b-ceph-3 ==
 mode lacp-static
 trunkport 10GE 1/0/3
 trunkport 10GE 2/0/3
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#
############## BMC接入交换机B-BMC-Leaf1（华为CE6881-48S6CQ） ################
#配置交换机名称
sysname B-BMC-Leaf1
#配置BMC管理VLAN
vlan batch 30 40
#
vlan 30
 description == Mgmt-Server-BMC ==
#
vlan 40
 description == Mgmt-Switch-BMC ==
#配BMC交换机带内管理
interface Vlanif40
 ip address 192.168.236.221 24
#
ip route-static 0.0.0.0 0.0.0.0 192.168.236.254
#
#配置上联核心B-Mgmt-Spine1端口
interface Eth-Trunk100
 description == To_B-Mgmt-Spine1 ==
 mode lacp-static
 trunkport 100GE 1/0/1
 trunkport 100GE 1/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 30 40
#
interface GE1/0/1
 description == To_B-Spine1A == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/2
 description == To_B-Spine1B == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/3
 description == To_B-Mgmt-Leaf1A == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/4
 description == To_B-Mgmt-Leaf1B == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/5
 description == To_B-Biz-Leaf1A == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/6
 description == To_B-Biz-Leaf1B == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/7
 description == To_B-Stor-Leaf1A == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/8
 description == To_B-Stor-Leaf1B == 
 port link-type access
 port default vlan 40
 stp edged-port enable
interface GE1/0/9
 description == To_b-zstack-1 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/10
 description == To_b-zstack-2 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/11
 description == To_b-zstack-3 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/12
 description == To_b-ceph-1 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/13
 description == To_b-ceph-2 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
interface GE1/0/14
 description == To_b-ceph-3 == 
 port link-type access
 port default vlan 30
 stp edged-port enable
#

SiteC配置参考：

############## 核心交换机C-Spine1（华为CE6881-48S6CQ） ################
#堆叠配置请参考厂商产品文档
#配置交换机名称
sysname C-Spine1
#配置生成树
stp root primary
stp tc-protection
stp bpdu-protection
#
#配置管理VLAN
vlan batch 10
#
vlan 10
 description == Mgmt-Server ==
#
interface Vlanif10
 ip address 192.168.220.254 24
#配置存储VLAN
vlan batch 90
#
vlan 90
 description == Storage ==
#
interface Vlanif90
 ip address 192.168.221.254 24
#
#配置下联服务器c-zstack-1管理端口
interface Eth-Trunk1
 description == To_c-zstack-1 ==
 mode lacp-static
 trunkport 10GE 1/0/1
 trunkport 10GE 2/0/1
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 10
 port trunk pvid vlan 10
 stp edged-port enable
#配置下联服务器c-zstack-1存储端口
interface Eth-Trunk1
 description == To_c-zstack-1 ==
 mode lacp-static
 trunkport 10GE 1/0/2
 trunkport 10GE 2/0/2
 port link-type trunk
 undo port trunk allow-pass vlan 1
 port trunk allow-pass vlan 90
 port trunk pvid vlan 90
 stp edged-port enable
#

Site互连配置参考：

############## SiteA核心交换机A-Spine1（华为CE12804） ################
interface Eth-Trunk101
 description == To_B-Spine1 ==
 mode lacp-static
 trunkport 10GE 1/3/0/1
 trunkport 10GE 2/3/0/1
 undo portswitch
 ip address 192.168.200.1 30
#
interface Eth-Trunk102
 description == To_C-Spine1 ==
 mode lacp-static
 trunkport 10GE 1/3/0/2
 trunkport 10GE 2/3/0/2
 undo portswitch
 ip address 192.168.200.5 30
#
ip route-static 192.168.220.0 22 192.168.200.6 description To_SiteC
#

############## SiteB核心交换机A-Spine1（华为CE12804） ################
interface Eth-Trunk101
 description == To_A-Spine1 ==
 mode lacp-static
 trunkport 10GE 1/3/0/1
 trunkport 10GE 2/3/0/1
 undo portswitch
 ip address 192.168.200.2 30
#
interface Eth-Trunk102
 description == To_C-Spine1 ==
 mode lacp-static
 trunkport 10GE 1/3/0/2
 trunkport 10GE 2/3/0/2
 undo portswitch
 ip address 192.168.200.9 30
#
ip route-static 192.168.220.0 22 192.168.200.10 description To_SiteC
#

############## SiteC核心交换机C-Spine1（华为CE6881-48S6CQ） ################
interface Eth-Trunk101
 description == To_A-Spine1 ==
 mode lacp-static
 trunkport 10GE 1/0/47
 trunkport 10GE 2/0/47
 undo portswitch
 ip address 192.168.200.6 30
#
interface Eth-Trunk102
 description == To_B-Spine1 ==
 mode lacp-static
 trunkport 10GE 1/0/48
 trunkport 10GE 2/0/48
 undo portswitch
 ip address 192.168.200.10 30
#
ip route-static 192.168.224.0 19 192.168.200.5 description To_SiteA 
ip route-static 192.168.224.0 19 192.168.200.9 description To_SiteB
ip route-static 192.168.224.0 25 192.168.200.5 description To_SiteA
ip route-static 192.168.252.0 25 192.168.200.5 description To_SiteA
ip route-static 192.168.224.128 25 192.168.200.9 description To_SiteB
ip route-static 192.168.252.128 25 192.168.200.9 description To_SiteB
#

results matching ""

    No results matching ""